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ABSTRACT 

We present results of a survey for Lya emitters at z w 4.86 based on optical narrowband (A c = 
7126A, AA = 73A) and broadband (B, V, r' , i', and z') observations of the Cosmic Evolution Survey 
(COSMOS) field using Suprime-Cam on the Subaru Telescope. We find 79 LAE candidates at z ~ 4.86 
over a contiguous survey area of 1.83 deg 2 , down to the Lya line flux of 1.47 x 10~ ergs s 



- 1 cm- 2 . We 

obtain the Lya luminosity function with a best-fit Schechter parameters of logL* = 42.9^° 3 er g s s_1 



and 4>* = 1.2+®'5 x 10 4 Mpc 3 for a = —1.5 (fixed). The two-point correlation function for our LAE 

sample is £(r) = (r/4.4t^Mpc)- 1 ' 90±0 ' 22 . 

In order to investigate the field-to-field variations of the properties of Lya emitters, we divide the 
survey area into nine tiles of 0.5° x 0.5° each. We find that the number density varies with a factor of 
~ 2 from field to field with high statistical significance. However, we find no significant field-to-field 
variance when we divide the field into four tiles with 0.7° x 0.7° each. We conclude that at least 0.5 
deg 2 survey area is required to derive averaged properties of LAEs at z ~ 5, and our survey field is 
wide enough to overcome the cosmic variance. 

Subject headings: galaxies: distances and redshifts — galaxies: evolution — galaxies: luminosity 
function, mass function 



1. INTRODUCTION 

Study of the formation and evolution of galaxies is 
among the most important topics in modern astro- 
physics. An essential component of such investigations 
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is the identification of galaxies at the highest redshifts, 
when most of the galaxies formed, and study of their rest- 
frame properties. This requires multi- waveband, wide- 
area and deep surveys of galaxies to provide statistically 
significant population of these objects. Recently, com- 
plementary observations of selected fields by the largest 
ground-based and space-borne telescopes have made this 
aim possible by extending this study to z ~ 7 and pro- 
viding statistically large samples of high redshift galaxies 
with a significant fraction of them confirmed spectroscop- 
ically (see Taniguchi 2008 for a recent review) . 

To summarize, there are two established techniques 
to search for high-z galaxy candidates. Firstly, the Ly- 
man break method (i.e. also called drop-out technique) 
which identifies the continuum break characteristic of 
Lyman alpha absorption by the Inter-Galactic Medium 
(IGM) (Steidel et al 1996; for a review see Giavalisco 
2002). The high-z candidates selected by this technique 
are called Lyman break galaxies (LBGs). Secondly, the 
narrow-band imaging technique which aims for detection 
of galaxies with strong Lyman a emission- so called Ly- 
man alpha emitters (LAEs) (Taniguchi et al. 2003 for a 
review). Although both LBGs and LAEs are actively 
star-forming galaxies, there are systematic differences 
between them. For example, the stellar populations of 
LAEs are relatively younger, they have a smaller stel- 
lar mass (e.g., Lai et al. 2008), smaller size (e.g., Dow- 
Hygelund et al. 2007) and are less dusty (e,g, Shapley 
et al. 2003) compared to the LBGs. These observations 
imply that the LAEs are likely to be in an earlier star 
formation phase with respect to LBGs. Furthermore, it 
is estimated that the average mass of dark matter halos 
hosting LAEs and LBGs at z ~ 4 - 5 (~ 1O 12 M ) are 
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comparable (Ouchi et al. 2004; Kovac et al. 2007), while, 
at z ~ 3.1 (~ lO n M ), it is smaller for the LAEs (Ga- 
wiser et al. 2007). This implies that the LAEs at z ~ 3.1 
are likely progenitors of present-day L* galaxies, whereas 
the LAEs at z ~ 4 - 5 and LBGs at z ~ 3 - 5 will evolve 
into present-day galaxies with L > 2.5L*(Gawiser et al. 
2007). 

To understand differences between the LAEs and LBGs 
at any given redshift and their properties with look-back 
time, one needs statistically large and complete samples 
of these galaxies at different redshifts. Specially for the 
LAEs, due to technical difficulties in performing narrow- 
band observations, the majority of these surveys are per- 
formed over small areas and in selected redshift slices 
where there are windows to avoid absorption of the lines 
by the atmosphere. This problem is particularly serious 
for candidates at higher redshifts where one needs both 
depth and wide-area coverage to have sufficient number 
of galaxies and to minimize the cosmic variance. 

For the LAEs at z ~ 5.7, extensive studies in differ- 
ent fields have been performed, including: survey around 
quasar SDSS J1044-0125 (Ajiki et al. 2003), SSA22 
(Hu et al. 2004), GOODS-N and GOODS-S (Ajiki et 
al. 2006), the Subaru Deep Field (SDF) (Shimasaku et 
al. 2006), the Subaru-XMM Newton Deep Field (SXDF) 
(Ouchi et al. 2005, 2008) and the Cosmic Evolution Sur- 
vey (COSMOS) (Murayama et al. 2007). However, there 
are only limited surveys of LAEs at other redshifts. This 
is a serious deficiency in studying evolution of clustering 
of the LAEs and their rest-frame properties specially if 
these are expected to evolve to nearby elliptical galaxies 
(Gawiser et al. 2007). 

In this paper we perform the largest survey of the LAEs 
at z « 4.86, covering the entire 2 square degree of the 
COSMOS field (Scoville et al. 2007). Earlier studies of 
the LAEs at this redshift revealed presence of large scale 
structures of 20 x 50 Mpc size (Ouchi et al. 2003; Shi- 
masaku et al. 2003), that are comparable to almost the 
size of the surveyed area, indicating serious cosmic vari- 
ance in these data (Shimasaku et al. 2003). The survey 
performed in this study covers an area of 190 Mpc x 190 
Mpc [7 times larger than the survey area of Shimasaku et 
al. (2003, 2004)], large enough to encompass structures 
of ~ 50 x 50 Mpc 2 size, allowing for proper sampling of 
the average properties of LAEs at z ~ 4.9. Therefore, 
we are able to examine how the cosmic variance affects 
the derivation of both the Lya luminosity functions and 
the clustering properties for the first time. 

In the next section we present our sample selection of 
LAEs. In section 3 we discuss the Lya luminosity func- 
tion and the clustering properties of our sample. We 
summarize our results in section 4. Throughout this 
paper, magnitudes are given in the AB system. We 
adopt a flat universe with f2 ma tter = 0.3, Cl\ — 0.7, and 
H = 70 kms" 1 Mpc" 1 . 

2. THE SAMPLE 
2.1. The Data 
We carried out an optical narrow-band (NB711; A c = 
7126A, AA = 73 A) imaging survey of the entire 2-deg 
area of the COSMOS field, using the Suprime-Cam on 
the Subaru Telescope. The NB711 observations were 
done on February 2006 (Taniguchi et al. 2008). The 



data were reduced using the IMCAT software. 20 Com- 
bining the NB711 data with the broad-band (B, V, r', i' , 
and z') Suprime-Cam imaging data and z*-band Mega- 
Prime/CFHT imaging data already available (Taniguchi 
et al. 2007; Capek et al. 2007), 21 we identified LAE 
candidates at z w 4.86. Details of the narrow-band 
and broad-band observations and data reduction are pre- 
sented by Taniguchi et al. (2007, 2008) and Capak et al. 
(2007). 

The FWHMs corresponding to the PSFs on the final 
images are 0'.'95 (B), 1'.'33 (V), l'/05 (/), 0'.'95 (£'), T/15 
(z f ) and 0'.'79 (NB711). The images were all degraded to 
a PSF size of 1'.'33. The limiting AB magnitudes of the 
final PSF matched images are: B = 27.56, V = 26.77, 
r' = 26.95, i' = 26.49, z' = 25.45, and NB711 = 25.17 
for a 3<7 detection in a 3" diameter aperture. We then 
performed source detection on the original NB711 image 
using SExtractor (Bertin & Arnouts 1996), followed by 
photometry over an aperture of 3" diameter as described 
in Capak et al (2007). Similarly, i* band (CFHT) mag- 
nitudes over the same aperture, are used to identify in- 
terlopers consisting of bright galaxies with i' < 22. 

After subtracting the masked-out regions, the effective 
survey area is 1.83 deg 2 . The redshift coverage of NB711 
is 4.83 < z < 4.89 (Az = 0.06), giving an effective survey 
volume of 1.1 x 10 6 Mpc 3 (comoving). 

2.2. Selection of Lyman a Emitters at z w 4.86 

In order to select NB711-excess objects efficiently, we 
first need the magnitude of a frequency-matched contin- 
uum. Since the effective frequency of the NB711 filter 
(421.1 THz) lies between r' (482.8 THz) and i' (394.0 
THz) bands, we estimate the frequency- matched con- 
tinuum, "rz continuum" , using the linear combination: 
f r i = 0.3/,' + 0.7/;', where f r i and fi> are the flux den- 
sities in r' and i' bands respectively. This gives a 3 a 
limiting magnitude of riu m .3a = 26.84 for the contin- 
uum (in a 3" diameter aperture). Since brighter objects 
with i' < 22 are saturated in Subaru images, we use the 
i* flux density, fi* , to calculate the "ri continuum" for 
such objects, i.e., /h = 0.3/ r ' + 0.7 fi*. 

The ./Vi?711-excess objects are then selected using the 
following criteria: 

ri- NB7ll>0.7, and (1) 
ri-NB711>3o-(ri-NB711), (2) 

where 3<r(ri — NB711) = 

-2-5 fog (l - V(f^B^) 2 + (f3a„) 2 /fNB71l) ■ 

The first criterion corresponding to an observed equiv- 
alent width of EW b s > 66 A, means the flux density 
in narrow band is twice as large as the flux density of 
ri continuum. This kind of criterion is conventionally 
used for LAE survey (e.g., Ouchi et al. 2003; Ajiki 
et al. 2006; Murayama et al. 2007) to select reliable 
emitter candidates. Taking account the scatter of the 
ri — NB711 color, we added the second criterion. These 
two criteria are shown in Figure [1] For objects with 
ri < ri\i m ^ a , we use the lower-limit value of the NB711- 
excess, (ri - iV.B711)iow.li m it = ri Um ^o- - NB711, for our 

20 IMCAT is distributed by Nick Kaiser at 
http:/ /www.ifa.ha,waii.cdu~kaiscr/imcat/ . 

zi http://irsa.ipac.caltech.edu/data/COSMOS/ 
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sample selection. We finally select the NB711-excess ob- 
jects with NBUl < 24.9. 

Following the above criteria, we find a total of 1154 
NB711-excess objects. These objects includes not only 
LAEs at z k 4.86, but other low-z interlopers such as Ha, 
[Oin], and [On] emitting galaxies. In order to distinguish 
LAEs from low-z interlopers, we compare the observed 
broad-band colors of the LAE candidates with colors that 
are estimated by using the model spectral energy distri- 
bution derived by Coleman, Wu, & Weedman (1980), 
Kinney et al. (1996), and Bruzual & Chariot (2003). 
Figure [2] shows the ri — NB711 vs. r' — i' color-color 
diagram with the predicted colors overlaid. Because of 
the cosmic transmission (Madau et al. 1996), the r' — i 1 
colors of LAEs are predicted to be redder than low-z 
emission-line galaxies. Based on results from Figure [2J 
we added another condition to the selection criteria: 

r' - i' > 0.8 . (3) 

Since the Lyman break is redshifted to ~ 5300 A, the 
B-band flux of LAEs at z « 4.86 is expected to be zero. 
The i?-dropout is an effective criterion to distinguish 
LAEs from low-z interlopers. Here, we must pay at- 
tention to the contamination from nearby objects on the 
sky. If there are objects detected in .B-band near the 
LAE and the fluxes from these objects in the aperture 
are not negligible, the LAE may be misclassified as a low- 
z interloper. We show B-band images of two of our final 
LAE candidates in Figure [H Although there is no object 
at the position of the emission-line object (center), the 
B-band magnitude in 3" diameter aperture is brighter 
than 27.56 (3cr), because of the contamination from the 
object that lie at a distance of ~ 1.5" from the center. 
To avoid the contamination from such nearby objects, we 
adopt the B-band magnitude measured with in the small 
aperture (O'/50) in the original image. We therefore add 
another selection criterion, 

£original(tf'5<£) > 30.09 , (4) 

where -B O riginai(0'.'5</>) is the B-band magnitude over a 
0'.'5 diameter aperture, measured in the original image 
(i.e. the image before convolving and with a PSF size of 
0'.'95). The 3cr limiting magnitude for a O'/50) aperture 
in the original image is 30.09. 

Based on the added criteria, we can now clearly distin- 
guish LAEs from the low-z interlopers. We finally select 
a total of 79 LAE candidates at z ~ 4.86. The photomet- 
ric properties of these LAE candidates are listed in Table 
1. Their broad- and NB711-band images are presented 
in Figure 3. 

To further check the validity of our photometric selec- 
tion and their expected redshifts, we extract information 
of the LAE candidates from the COSMOS spectroscopic 
catalogue. A total of 7 LAEs in our final candidate list 
have spectroscopic redshifts. Figure 4 presents the spec- 
troscopic redshift distribution of these LAEs. This peaks 
at z w 4.85 with all the spectroscopic redshifts lying in 
the range 4.80 < z < 4.85. This result suggests that 
our selection criteria works quite well to identify LAEs 
atzw 4.9. 

2.3. Lya Luminosity 

We estimate the line fluxes for our LAE candidates, 
FL, using the prescription by Pascual et al. (2001). We 



express the flux density in each filter band using the line 
flux, Fl, and the continuum flux density, fc- 



/nb — fc + 

fr' = fc 

U=fc + 



F r . 



ANB 



Fl_ 
Ai'' 



(5) 
(6) 
(7) 



where ANB and Ai' are the effective bandwidths of 
NB711 and i', respectively. The ri continuum is ex- 
pressed as 

f H = 0.3/ P » + 0.7 U =fc + 0.7^y. (8) 

Ai' 

Using equations © and (©, the line flux, Fl, is calcu- 
lated by 



F L = ANB 



/NB - fr 



1 -0.7(ANB/Ai')' 



(9) 



The limiting line flux of our survey is 1.47 x 
10~ 17 crgs s -1 . Since the response curve is not square 
in shape, the observed flux of Lya emission for a fixed 
Lya luminosity depends on the redshift. On average, the 
observed flux is underestimated by a factor of 0.81, which 
is calculated by 



A c +AA/2 



rA c -t 

Ja,- 



= 0.81 



X c -AX/2 



dX 



where R{\) is a response function normalized by the 
maximum value. We therefore apply correction statis- 
tically for the filter response by F cor (Lya) — Fl x 1.24. 
Finally, we estimate the Lya luminosity as L(Lya) = 
47rdLF cor (Lya). In this procedure, we assume that all 
the LAEs are located at z = 4.86 (g?l = 45.1 Gpc), the 
redshift corresponding to the central wavelength of our 
NB711 filter. 

3. RESULTS AND DISCUSSION 
3.1. Spatial Distribution 

Figure[6]presents the spatial distribution of our 79 LAE 
candidates_ at z ~ 4.86. The contours of local surface 
density (2E, where E is the averaged surface density over 
the whole field, 43 degree -2 ) are shown in the figure. 
The local surface density at position (x, y) is the density 
averaged over the circle centered at (x, y), whose radius 
is determined as the angular distance to the 3rd nearest 
neighbors. There are ten overdensity regions in the field. 
A typical size of the large overdensity region is 0.4° x 0.2° 
(50 Mpc x 25 Mpc), being similar to those found by 
Shimasaku et al. (2003) and Ouchi et al. (2005). 

To check for field-to-field variation, we divide the sur- 
vey area into nine subfields, each corresponding to a sky 
area of 0.5° x 0.5° (63 Mpc x 63 Mpc) (Figure©. The 
number density of the LAEs in each subfield is summa- 
rized in Table [U We find significant field-to-field varia- 
tions among the nine subfields by a factor of ~ 2. This 
means that the typical scale of the large scale structure 
is comparable to the size of the subfield, that is consis- 
tent with the size of the overdensity regions found in the 
above. The field-to-field variations found here, agrees 
with those for LAEs at z ~ 5.7, independently estimated 
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in the SXDF (Ouchi et al. 2008) and with theoretical 
predictions using the cosmological hydrodynamic simu- 
lations (Nagamine et al. 2008) for the fields of ~ 0.2deg 2 . 
Our finding suggests that the derived properties of LAEs 
from the survey with a small survey area (smaller than 
0.5° x 0.5°) may be affected by the cosmic variance. 

We also divide the survey area into four subfields, each 
corresponding to a sky area of 0.7° x 0.7° (95 Mpc x 95 
Mpc). We find 21, 21, 19 and 18 LAEs in NE, NW, 
SW, and SE quadrant, respectively. This means that 
the typical scale of the large scale structure is smaller 
than the size of the subfield. Our finding suggests that 
the derived properties of LAEs from the survey with a 
large survey area (larger than 0.7° x 0.7°) are considered 
to be averaged ones over the universe at z ~ 5. 

3.2. Lya Luminosity Function 

The rest-frame Lya luminosity function (LF) for our 
sample of LAEs at z ks 4.86 is presented in Figure[7l The 
LF is measured as 

$QogLi) = (10) 

where V co is the comoving volume of 1.1 x 10 6 Mpc 3 
(4.83 < z < 4.89) and 2Vj is a number of LAEs within 
logi, ± ±AlogL. We use AlogL(Lya) = 0.2. We 
fit the rest-frame Lya LF with the Schechter function 
(Schechter 1976) using parametric maximum likelihood 
estimator (Sandage, Tammann, & Yahil 1979). Since the 
characteristic luminosity (L*) and the faint-end slope (a) 
of the Schechter LFs are not independent, we perform 
the fit by fixing a to —1, —1.5, and —2. Our best-fit 
Schechter parameters are summarized in Table [3] For 
comparison, we also plot the Lya LF for a sample se- 
lected by Ouchi et al. (2003). Their survey was per- 
formed by using the same narrowband filter (NB711), 
for smaller field (543 arcmin 2 ) and deeper (down to 
NB711 = 26.0) than ours. Although our sample does 
not include low-luminosity LAEs and their sample does 
not include LAEs at the luminous-end, our Lya LF is 
consistent with that of Ouchi et al. (2003) for the range 
of 42.8 < logi(Lya) < 43.2. 

Since the filter response curve of NB711 is not box- 
shaped, the narrow-band magnitude of LAEs of a fixed 
Lya luminosity varies as a function of the redshift. The 
selection function of LAEs in terms of the equivalent 
width also changes with the redshift (Shimasaku et al. 
2006; Ouchi et al. 2008). We check the validity of the 
Lya LF derived above. In order to examine whether 
or not we can reconstruct an input Lya LF by our selec- 
tion criteria and an estimation of Lya flux, we performed 
the Monte Carlo simulations that are similar to those 
made by Shimasaku et al. (2006). First, we generate a 
mock catalog of LAEs with a set of the Schechter param- 
eters (a, 0*, L*) and a Gaussian distribution function of 
EW, f(EW )dEW oc exp(-£W 2 /2a EW )dEW . We 
adopt four gew values: oew — 50, 100, 200, and 400 
A. We uniformly distribute them in comoving space over 
4.7 < z < 5.1. Next, we select Lya emitters and evalu- 
ate the Lya LF applying the method written above for 
the mock catalog. We show results of our simulations in 
Figure [H We confirm that the Lya LF we evaluate is 
very close to the input LF. We conclude that the simple 
method we adopted is valid for evaluating the Lya LF. 



We plot the Lya LFs from the four subfields in Figure[7] 
(left panel). Those LFs are consistent within their errors. 
We also summarize the best-fit Schechter parameters for 
four subfields in Table [4] Although the field-to-field vari- 
ation of (f>* is a factor of 4, each value exist within the 
error in Table [3l In Figure [7] (right panel), we compare 
our results with other LAE surveys in the redshift range 
z ~ 3.1 - 6.6. Although various surveys have slightly 
different selection criteria, most of the Lya LFs are sim- 
ilar to each other. We then find that estimated Lya LF 
is very similar to those at 3.1 < z < 5.7 within errors. 
This result supports the little evolution of Lya LFs in 
the range of 3 < z < 6 (Tran et al. 2004; van Breukelen 
et al. 2005; Shimasaku et al. 2006; Ouchi et al. 2008). 

3.3. Equivalent Widths 

Figure [9] shows the distribution of EWo(Lya). To mea- 
sure the rest-frame UV continuum flux, we use the z'- 
band data as the fluxes at i'-band are affected by Lya 
emission. For objects fainter than la in the z' band, we 
calculate the upper-limit of the UV luminosity, L V (UV), 
and the lower-limit of the rest-frame equivalent width, 
EWo(Lya). The EW^iLya) distribution is similar to 
those in previous studies of LAEs at z ~ 3-6 (e.g., Shi- 
masaku et al. 2006; Dawson et al. 2007; Ouchi et al. 
2008; Gronwall et al. 2008), with the mean rest-frame 
Lyman a equivalent widths of the sample smaller than 
200 A. There is no LAE with EW (Lya) > 250 A in our 
sample, although the rest-frame Lya equivalent widths of 
23 of the LAEs in our sample (29%) are lower limits. Tak- 
ing account of a predicted EWq (Lya) for starburst galax- 
ies, 300 A for young starburst (age < 10 6 yr) and 100 
A for old starburst (age ~ 10 8 yr) (Malhotra & Rhoads 
2002), we consider that there is no peculiar object in our 
sample. Figure [TOl shows the relation between EWo(Lya) 
and Myv- There is no object with E Wq (Lya) > 80 A in 
the UV-bright (Muv < -21.5) sample. Although the 
number of UV-bright LAEs is small and the uncertain- 
ties on EWqS for UV-faint objects are large, this trend 
is similar to that found for LBGs and LAEs at z ~ 5-6 
(Ando et al. 2006; Shimasaku et al. 2006; Ouchi et al. 
2008). We conclude that our sample shows the "average" 
picture of bright LAEs at z ~ 5. 

3.4. UV Luminosity Function 

Figure [TT1 shows UV LF of our sample (black symbols). 
The UV LFs of LAEs estimated in the four subfields are 
consistent within errors. Figure QT] also include the UV 
LF of LBGs at z ~ 5 (Yoshida et al. 2006) and LAEs at 
z ~ 3.1, 3.7, and 5.7 (Ouchi et al. 2008). The shape of 
our UV LF seems different from those of previous works 
and is not fit by Schechter function, since a detection 
limit of rest-frame equivalent width, EWq (Lya), depends 
on M uv , e.g., EW (Lya) > llA at M uv < -21.5 and 
EW (Lya) > 57A at M uv = -20 (see Figure [TO]). As 
a reference, we overlay the result of our Monte Carlo 
simulation for a = —1.5 and o~ew — 100 A: dotted line 
show input UV LF for EWo(Lya) > 11 A and solid line 
show output UV LF. This result also shows that our UV 
LF is considered to be complete for LAE with EWq > 
llA for Myv < —21.5. We therefore concentrate the 
number density at Muv < —21.5. 

First, we compare our UV LF with that of LBGs at 
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z ~ 5. The number density of our LAEs is comparable 
to that of LBGs at z ~ 5 at Muv ~ -22 and ~ 20- 
25 % at M uv = -21.5. Ouchi et al. (2008) pointed 
out that the ratio of number densities of LAEs to those 
of LBGs is - 10% at z = 3-4 and > 50% at z = 5.7. 
Our result imply that the ratio of the number density 
of LAEs to that of LBGs becomes larger with redshift 
from z = 4 to 5. Next, we compare our UV LF with 
those of LAEs at different redshifts. Figure [12] shows the 
number density of LAEs at Muv = —21.5 as a function 
of z. The number density of our LAEs at Muv = —21.5 
is comparable to that of LAEs at z ~ 5.7, while larger 
than those of LAEs at z ~ 3.1 and 3.7. The number 
density of UV-bright LAEs (Muv < —21.5) increases an 
order of magnitude with redshift from z — 4 to 5. Since 
it is likely that the LAEs are star-forming galaxies in an 
earlier star formation phase, our findings imply that the 
initial active star-formation phase occur mainly beyond 

2 = 5. 

3.5. Clustering Properties 

We found the large scale structure of LAEs of 0.4° x 
0.2° in subsection [3TTJ In order to perform a more quan- 
titative study of the clustering properties of the LAEs 
at z ~ 4.86, we derive their angular two-point correla- 
tion function (ACF), w(8), using the estimator defined 
by Landy & Szalay (1993), 

where DD(9), DR(9), and RR(9) are normalized num- 
bers of galaxy-galaxy, galaxy-random, and random- 
random pairs, respectively. The random sample here 
consists of 100,000 sources with the same geometri- 
cal constraints as the galaxy sample. The observed 
ACF is fit well by a single power law: w(9) = 
0.021±°;°^ 6>-°- 90±0 - 22 (Figure E). The correlation 
length, ro, is calculated from the ACF through Limber's 
equation (e.g., Peebles 1980), assuming a top-hat red- 
shift distribution centered on z = 4.86 ± 0.03. We es- 
timated the ro corresponding to our sample of LAEs as 
r — 4.4^2 g Mpc. The two-point correlation function is 

thus written as £(r) = (r/4.4±lj;£ Mpc)- 1 - 90 ^- 22 . This 
agrees well with results from other works at similar red- 
shifts, e.g., r = 5.0 ±0.4 for z ~ 4.9 (Ouchi et al. 2003); 
r = 4.57 ± 0.60 for z ~ 4.5 (Kovac et al. 2007). 

Also shown in Figure [T3] are the ACFs for the LAEs 
in the four subfields. We detect strong clustering sig- 
nals in small scale (8 < 50 arcsec) for NE, SW, and SE 
subfields, with the NW subfield showing no clustering 



signals at any angular separations. Although this may 
imply the presence of a cosmic variance on the cluster- 
ing properties similar to that found in a previous study 
(Shimasaku et al. 2004), taking account of large uncer- 
tainties of ACFs, we consider that there are no significant 
field-to-field variations among the four subfields. 

4. SUMMARY 

We have performed the largest survey to date for Lya 
emitters at z » 4.86, using narrow-band (NB711) imag- 
ing technique in the COSMOS 2 square degree field. We 
have found a total of 79 Lya emission-line galaxy candi- 
dates. For 7 LAE candidates with available spectroscopic 
data, we have confirmed that our criteria for selecting 
LAEs at z » 4.86 are working well. Our results and 
conclusions are summarized below, 

1. We have found a field-to-field variation of the num- 
ber density of LAEs as large as a factor of ~ 2 among the 
nine subfields with 0.5° x 0.5°. On the other hand, the 
number density of LAEs for four subfields with 0.7° x 0.7° 
is consistent within a error. This finding is consistent 
with the scale of large scale structure we found, 50 Mpc 
x 25 Mpc. Wc conclude that at least 0.5 deg 2 survey 
area is required to derive averaged properties of LAEs at 
z ~ 5, and our survey field is wide enough to overcome 
the cosmic variance. 

2. The Lya LF is well-fitted by a Schechter func- 
tion with best-fit Schechter parameters: logL* — 



42.9ltg;t?ergs s" 1 and 



1.22 



+8.02 

-1.05 



x 10- 4 Mpc- J for 



a = —1.5 (fixed). The two-point correlation function is 
well fitted by a power law, w{9) = 0.021±™ 2 ^- - 90±0 - 22 , 

giving f (r) = (r/4.4i^Mpc)- 1 - 9 . 

3. We have derived the UV LF of LAEs. The number 
density of our LAEs at Muv = —21.5 are similar to those 
of LAEs at z ~ 5.7 while larger than those of LAEs 
at z ~ 3-4. The number density of UV-bright LAEs 
increases an order of magnitude with redshift from z ~ 4 
to z ~ 5. 
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TABLE 1 

A LIST OF LYQ EMITTER CANDIDATES. 



# 


RA 


DEC 


r' i' ri NB711 


z> 


logL(Lya) logL l/ (] 


.540A) M uv 


EW 




(deg) 


(deg) 


(mag) (mag) (mag) (mag) 


(mag) 


(crgss -1 ) (ergs s _1 


Hz" 1 ) (mag) 


(A) 



1 


150 


.68983 


1 


598039 


28. 


85 


26 


.54 


26. 


.87 


2 


150 


.58466 


1 


.528353 


26. 


26 


25 


.13 


25. 


.36 


3 


150 


.43377 


1 


.584748 


27. 


49 


26 


.38 


26. 


61 


4 


150 


.47017 


1 


.527121 


27. 


85 


26 


.69 


26 


92 


5 


150 


.12679 


1 


.606008 


27. 


15 


25 


.63 


25. 


.91 


6 


150 


.19137 


1 


.514911 


25. 


.95 


25 


.13 


25. 


.32 


7 


149 


.42627 


1 


570369 


27. 


34 


25 


.94 


26. 


21 


8 


149 


.50750 


1 


.569846 


26. 


40 


24 


.72 


25. 


.01 


9 


150 


.72870 


1 


.654431 


27. 


26 


26 


.02 


26. 


.27 


10 


150 


.69867 


1 


.658967 


99. 


00 


27 


,04 


27. 


.42 


11 


150 


.69845 


1 


.643227 


27. 


19 


25 


.73 


26 


00 


12 


150 


.44771 


1 


.639259 


26. 


33 


24 


.16 


24. 


.48 


13 


150 


.21979 


1 


.647579 


27. 


64 


25 


.97 


26. 


.26 


14 


149 


.92387 


1 


706955 


27. 


91 


26 


.99 


27. 


19 


15 


149 


.86911 


1 


.741172 


27. 


83 


25 


.18 


25 


.52 


16 


149 


,81740 


1 


.738043 


26. 


33 


25 


.43 


25. 


.63 


17 


149 


.85339 


1 


702846 


28. 


17 


26 


.76 


27. 


03 


18 


149 


.81761 


1 


.638761 


26. 


26 


25 


.2.3 


25 


.45 


19 


149 


.47913 


1 


.713145 


26. 


.26 


24 


.90 


25 


.17 


20 


150 


.77783 


1 


795379 


27. 


63 


25 


.85 


26. 


15 


21 


150 


.43713 


1 


.821238 


27. 


16 


25 


.26 


25. 


.57 


22 


150 


.39265 


1 


.852772 


30. 


47 


26 


.36 


26. 


.74 


23 


150 


.31602 


1 


848847 


26. 


35 


24 


.94 


25. 


21 


24 


149 


.98396 


1 


.914333 


27. 


61 


26 


.12 


26. 


39 


25 


149 


.76505 


1 


835950 


26. 


46 


25 


.04 


25 


31 


26 


149 


.82192 


1 


.826156 


27. 


56 


26 


.12 


26. 


39 


27 


149 


.70144 


1 


880336 


29. 


66 


25 


.88 


26 


26 


28 


149 


.43575 


1 


.958916 


25. 


53 


24 


.38 


24. 


62 


29 


150 


.52280 


2 


.053999 


28. 


66 


26 


.82 


27. 


13 


30 


150 


.44155 


2 


045647 


25. 


92 


24 


.17 


24, 


,47 


31 


149 


.80258 


1 


.976421 


28. 


37 


26 


.13 


26. 


46 


.32 


149 


.47068 


2 


.112708 


28. 


53 


27 


.11 


27. 


38 


.33 


149 


.49438 


2. 


111401 


27. 


52 


26 


.13 


26 


39 


34 


149 


.50653 


2. 


059920 


26. 


21 


24 


.78 


25 


05 


35 


149 


.42625 


1 


.971732 


27. 


00 


25 


.60 


25. 


87 


36 


150 


.75362 


2. 


237688 


27. 


53 


26 


.18 


26 


.45 


37 


150 


.74435 


2. 


216502 


25. 


70 


24 


.59 


24. 


,82 


38 


150 


.23097 


2. 


219221 


27. 


9.3 


26 


.37 


26 


65 


.39 


150 


.17687 


2 


.162903 


27. 


02 


26 


.05 


26. 


26 


40 


149 


.96795 


2. 


.258172 


27. 


64 


26 


.05 


26. 


.34 


41 


150 


.01739 


2. 


.146056 


26. 


51 


24 


.88 


25. 


17 


42 


149 


.83435 


2 


.270296 


26. 


92 


25 


,37 


25 


65 


43 


150 


.68548 


2 


422582 


26. 


74 


25 


.83 


26 


03 


44 


150 


.48986 


2 


405317 


26. 


14 


24 


.93 


25 


17 


45 


150 


.34351 


2 


.380535 


26. 


92 


26 


.01 


26. 


22 


46 


150 


.17116 


2 


.443712 


27. 


53 


25 


.48 


25 


79 


47 


149 


.95843 


2 


414291 


29. 


20 


26 


.60 


26. 


.95 


48 


149 


.86004 


2 


.390346 


27. 


06 


26 


.09 


26 


31 


49 


149 


.62681 


2 


.428601 


31. 


96 


26 


.72 


27. 


11 


50 


149 


.51027 


2 


301385 


27. 


33 


26 


.22 


26 


.45 


51 


150 


.72903 


2 


.584166 


26. 


45 


25 


.56 


25. 


76 


52 


150 


.78495 


2 


.573355 


26. 


58 


25 


.42 


25. 


66 


53 


150 


.75115 


2 


.481606 


28. 


43 


26 


.25 


26. 


,57 


54 


150 


.24314 


2 


530345 


27. 


07 


25 


.42 


25 


71 


55 


150 


.13505 


2 


.486044 


26. 


8.3 


25 


.21 


25. 


50 


56 


149 


.89657 


2 


527743 


26. 


57 


25 


.42 


25 


66 


57 


149 


.75765 


2 


572967 


28. 


44 


26 


.53 


26 


84 


58 


149 


.87223 


2 


.497300 


27. 


31 


25 


.72 


26 


00 


59 


149 


.60335 


2 


.612591 


26. 


85 


25 


.52 


25 


78 


60 


149 


.58816 


2 


521003 


27. 


14 


25 


.43 


25 


73 


61 


149 


.46094 


2 


.563734 


27. 


24 


26 


.09 


26. 


33 


62 


150 


.80329 


2 


.730062 


26 


73 


25 


.50 


25. 


75 


63 


150 


.76442 


2 


.688660 


25. 


39 


24 


.31 


24. 


.53 


64 


150 


.43973 


2. 


720629 


26. 


24 


25 


.09 


25 


.33 


65 


150 


.30282 


2 


.772591 


26. 


65 


25 


.06 


25 


34 


66 


150 


.30023 


2 


666173 


28. 


11 


26 


.84 


27 


09 


67 


150 


.28152 


2. 


.651694 


27. 


53 


26 


.16 


26 


42 


68 


150 


.29721 


2 


.634812 


26 


25 


24 


.65 


24. 


94 


69 


149 


.94445 


2 


.704370 


26. 


02 


25 


.20 


25 


39 


70 


149 


.78731 


2 


.678302 


26. 


12 


24 


.67 


24. 


.94 


71 


149 


.66107 


2 


.739789 


26. 


23 


24 


.87 


25. 


14 


72 


149 


.72632 


2 


.664706 


99. 


00 


25 


.59 


25 


98 


73 


149 


.43064 


2 


784033 


28. 


95 


26 


.30 


26 


65 


74 


149 


.41782 


2 


735198 


26. 


83 


25 


.45 


25 


.71 


75 


149 


.44796 


2 


.694757 


26 


55 


25 


.36 


25. 


60 


76 


150 


.31928 


2 


864155 


27. 


78 


26 


.37 


26 


64 


77 


150 


.23253 


2 


849228 


99. 


00 


26 


.19 


26 


58 


78 


149 


.80660 


2 


.861745 


25. 


88 


25 


.05 


25. 


24 



,86 


25.95 


42.68 


28.80 


-20.4 


46 


,52 


25.40 


43.20 


29.02 


-21.0 


90 


,46 


> 26.64 


42.85 


< 28.52 


> -19.7 


> 127 


,76 


26.28 


42.73 


28.67 


-20.1 


69 


.43 


25.61 


43.28 


28.94 


-20.7 


131 


.47 


24.79 


42.64 


29.26 


-21.6 


14 


.44 


> 26.64 


42.83 


< 28.52 


> -19.7 


> 121 


.05 


25.06 


42.85 


29.15 


-21.3 


30 


.22 


> 26.64 


42.94 


< 28.52 


> -19.7 


> 156 


.80 


> 26.64 


42.74 


< 28.52 


> -19.7 


> 98 


.55 


24.98 


42.75 


29.19 


-21.4 


22 


.64 


24.31 


42.98 


29.46 


-22.0 


20 


.71 


25.67 


42.70 


28.91 


-20.7 


36 


.22 


> 26.64 


42.98 


< 28.52 


> -19.7 


> 171 


.07 


25.43 


42.94 


29.01 


-20.9 


51 


.46 


25.87 


42.73 


28.83 


-20.5 


47 


.74 


> 26.64 


42.75 


< 28.52 


> -19.7 


> 100 


.10 


25.25 


42.91 


29.08 


-21.1 


41 


.18 


24.82 


42.81 


29.25 


-21.5 


21 


.38 


25.44 


42.85 


29.00 


-20.9 


42 


.96 


24.83 


43.00 


29.25 


-21.5 


34 


.51 


> 26.64 


42.84 


< 28.52 


> -19.7 


> 122 


.41 


24.73 


42.65 


29.29 


-21.6 


14 


.49 


> 26.64 


42.82 


< 28.52 


> -19.7 


> 119 


.38 


24.49 


42.71 


29.38 


-21.9 


13 


.46 


> 26.64 


42.83 


< 28.52 


> -19.7 


> 122 


.38 


25.73 


42.86 


28.89 


-20.6 


56 


.34 


24.16 


43.20 


29.51 


-22.2 


29 


.87 


> 26.64 


43.13 


< 28.52 


> -19.7 


> 240 


.50 


24.13 


43.07 


29.53 


-22.2 


21 


.58 


25.48 


42.78 


28.99 


-20.9 


37 


.84 


> 26.64 


42.72 


< 28.52 


> -19.7 


> 94 


.64 


> 26.64 


42.75 


< 28.52 


> -19.7 


> 100 


.56 


25.14 


43.15 


29.12 


-21.2 


63 


.59 


25.36 


43.21 


29.04 


-21.0 


89 


.37 


26.27 


42.88 


28.67 


-20.1 


97 


.08 


24.31 


42.76 


29.45 


-22.0 


12 


.04 


> 26.64 


43.04 


< 28.52 


> -19.7 


> 196 


.21 


> 26.64 


42.95 


< 28.52 


> -19.7 


> 157 


.76 


26.26 


42.68 


28.68 


-20.1 


60 


.39 


25.62 


43.25 


28.93 


-20.7 


124 


.82 


25.58 


43.08 


28.95 


-20.8 


81 


.49 


25.59 


42.78 


28.94 


-20.8 


41 


.91 


25.16 


42.97 


29.11 


-21.2 


43 


.62 


26.54 


42.74 


28.56 


-19.8 


89 


.65 


25.27 


42.66 


29.07 


-21.1 


23 


.70 


26.49 


42.76 


28.58 


-19.9 


89 


.95 


> 26.64 


43.07 


< 28.52 


> -19.7 


> 208 


.93 


25.84 


42.66 


28.84 


-20.5 


40 


.69 


26.34 


43.19 


28.64 


-20.0 


208 


.64 


25.67 


42.65 


28.91 


-20.7 


33 


.62 


25.31 


42.64 


29.06 


-21.0 


23 


.41 


> 26.64 


42.87 


< 28.52 


> -19.7 


> 133 


,15 


25.44 


43.26 


29.00 


-20.9 


108 


.50 


25.22 


42.68 


29.09 


-21.1 


23 


.45 


26.08 


42.75 


28.75 


-20.3 


59 


.67 


25.54 


42.77 


28.96 


-20.8 


38 


.90 


25.50 


43.07 


28.98 


-20.9 


74 


.52 


26.03 


42.73 


28.77 


-20.3 


55 


.14 


25.28 


42.93 


29.07 


-21.1 


43 


.80 


25.71 


42.66 


28.90 


-20.6 


35 


.59 


25.90 


42.68 


28.82 


-20.5 


43 


.68 


23.91 


42.96 


29.62 


-22.4 


13 


.27 


25.30 


42.78 


29.06 


-21.1 


32 


.41 


24.81 


42.70 


29.26 


-21.5 


16 


.88 


> 26.64 


42.69 


< 28.52 


> -19.7 


> 87 


.16 


> 26.64 


42.98 


< 28.52 


> -19.7 


> 170 


.06 


25.12 


42.82 


29.13 


-21.2 


29 


.49 


25.79 


42.66 


28.86 


-20.6 


37 


.50 


24.88 


43.16 


29.23 


-21.5 


52 


.32 


24.66 


42.69 


29.31 


-21.7 


14 


.35 


> 26.64 


42.85 


< 28.52 


> -19.7 


> 126 


.81 


> 26.64 


42.69 


< 28.52 


> -19.7 


> 87 


.41 


> 26.64 


42.78 


< 28.52 


> -19.7 


> 107 


.54 


24.89 


42.68 


29.22 


-21.5 


17 


.66 


> 26.64 


42.76 


< 28.52 


> -19.7 


> 102 


.70 


> 26.64 


42.74 


< 28.52 


> -19.7 


> 97 


.25 


24.83 


43.33 


29.25 


-21.5 


71 



TABLE 2 

The number density of LAEs in the 9 subfields of 0.5° x 0.5°. 





East 

(deg- 2 ) 




Middle 

(deg- 2 ) 


West 
(deg- 2 ) 


North 

Middle 

South 


29.6 ±12.1 
30.9 ± 12.6 
59.5 ± 17.9 




52.0 ± 15.7 
28.0 ± 11.4 
28.0 ± 11.4 


64.3 ± 17.8 
45.3 ± 15.1 
53.1 ± 16.0 


TABLE 3 

Best-fit Schechter parameters for Lyq? luminosity functions. 


a 

(fixed) 




(erg s 1 ) 




(xl0- 4 Mpc- 3 ) 


-1.0 
-1.5 
-2.0 




42.821^ 
42.91±°j? 
43.00±°™ 




i 41+6.73 

1 99 +8.02 
-1.05 

0.82t 9 9 7 s 7 


TABLE 4 

Best-fit Schechter parameters for Lyo luminosity functions for each subfield. 


a 

(fixed) 


subfield 




(erg s L ) 


(xl0- 4 Mpc- 3 ) 


-1.0 


NE 
NW 
SW 
SE 




42.79 
42.76 
42.92 
42.83 


0.85 
3.0 

0.84 
1.4 


-1.5 


NE 
NW 
SW 
SE 




42.87 
42.84 
43.02 
42.91 


0.78 
2.7 

0.68 
1.2 


-2.0 


NE 

NW 
SW 
SE 




42.97 
42.94 
43.15 
43.02 


0.56 
2.0 
0.40 
0.83 
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Fig. 1. — ri - NB7U vs. NB7U diagram for all the NB711 detected sources in COSMOS. The horizontal solid line corresponds to 
ri — NB7U = 0.7. The dashed line show the distribution of the 3<r errors. The dotted line shows the limiting magnitude of ri. Filled 
circles represent 79 LAE candidates detected here. 




Fig. 2. — ri — NB711 vs. r'—i' diagram. Thick gray lines show colors of model LAE SEDs, which are calculated with BC03 model (Bruzual 
& Chariot 2003) with an exponential decay time of tau=lGyr and an age of lGyr, corresponding to a cosmic transmission of 0.5r c ff (left) 
and T e ff (right) in the formulation of Madau ct al. (1996). Luminosity of the Ly« emission is calculated as L(Lya) = 1.2 X 10 _11 A r Ly C , 
where Ni, yc is the ionizing photon production rate (Leitherer & Heckman 1995; Brocklhurst 1971). Thin gray lines show color loci of 
starburst galaxies (Kinney et al. 1996), typical elliptical, spiral, and irregular galaxies (Coleman, Wu, & Weedman 1980) up to z = 2. 




Fig. 3. — B-band images of our LAE candidates #16 & #44. Each box is 12" on a side (north is up and east is left). The diameter of 
a small (large) circle is (/.'5 (3"). In both cases, there are no counter part in the center, although the flux within 3" diameter aperture is 
larger than 3a because of the contamination from nearby objects. 
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Fig. 4.— 



Broad-band and NB711 images of our LAE candidates at z f» 4.9. Each box is 12" on a side (north is up and east is 




Fig. 4b. — continued. 




Fig. 4c. — continued. 
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Fig. 4d. — continued. 




Fig. 4e. — continued. 




Fig. 4f. — continued. 




Fig. 4g. — continued. 




Fig. 4h. — continued. 
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Fig. 5. — Spectroscopic redshift distribution of our LAE sample (7 LAEs). The dotted line shows the response function of the NB711 
band. 
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Fig. 6. — Spatial distributions of LAEs (black filled circles). The shaded regions show the areas masked out for the detection. The 
contours show the local surface density of the LAEs, drawn at the level twice as high as the average over the field, 43 dcg~ 2 . 




log L (Lyo) (ergs s 1 ) log L(Lya) (ergs s 1 ) 

Fig. 7. — Left: The Lya LF of our LAE sample (black symbols). The Lyo LF for the whole sample is shown with filled circles. The 
dotted, solid and dashed lines show the best-fit Schechtcr functions for the whole sample for a = —1, —1.5 and —2, respectively. The Lyo 
LFs for different quadrants are shown with boxes, diamonds, circles, and crosses for the NE, NW, SW, and SE subfield, respectively. For 
comparison, the Lya LF derived by Ouchi et al. (2003) is shown with inverse triangles. Right: Same as the left panel, compared with other 
surveys (gray symbols): for LAEs at z ~ 3.1 (Ouchi et al. 2008 = circles), z ~ 3.4 (Cowie & Hu 1998 = boxes), z ~ 3.7 (Ouchi ct al. 2008 
= triangles), z ~ 4.9 (Ouchi ct al. 2003 = inverse triangles), z ~ 5.7 (Rhoads & Malhotra 2001 = filled circles; Ajiki ct al. 2003 = filled 
boxes; Hu et al. 2004 = filled diamonds; Ajiki et al. 2006 = filled triangles; Shimasaku et al. 2006 = filled inverse triangles; Murayama et 
al. 2007 = half-filled circles; Ouchi et al. 2008 = half-filled boxes), and z ~ 6.6 (Taniguchi et al. 2005 = diamonds). 
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Fig. 8. — Results of our Monte Carlo simulations for a = —1.5. The derived Lyo luminosity functions are shown as thick solid line, 
thick dotted line, thick dashed line, thick dash-dotted line for the case of a^w = 50, 100, 200, & 400 A, respectively. These luminosity 
functions are similar to the input Schechter function (thin solid line). 
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Fig. 9. — Top: Distribution of the rest-frame Lya equivalent widths. Filled bars show the LAEs with the continuum detected above 
1<t. Open bars show the LAEs with no continuum detection. Bottom: Distribution of the rest-frame Lyo equivalent widths of the LAEs 
at z ~ 3.1 obtained by Gronwall et al. (2008). 




Fig. 10. — Rest-frame EWs of Lya lines vs. absolute magnitude at rest-frame 1540 A for our sample of LAEs at z ~ 4.9. Dashed 
lines show loci of the constant Lya luminosities for logL(Lya) = 43.6, 43.0 and 42.0, where L(Lya) is in units of ergs s _1 . Dotted line 
corresponds to Muv = —19.71 which is the rest-UV absolute magnitude corresponding to the z'-band limiting magnitude (lu). 
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Fig. 11. — The rest-frame UV LF of our LAE sample (black symbols). The UV LF for our whole sample is shown with filled circles. 
The UV LFs for different quadrants of the COSMOS field arc shown with black boxes, black diamonds, black circles, and black crosses 
for the NE, NW, SW, and SE field, respectively. The results of our Monte Carlo simulation for a = —1.5 and <jew = lOOAare overlaid: 
a dotted line show the input UV LF with EWo(Lya) > 13Aand a solid line show the output UV LF. For comparison, we show UV LFs 
from the previous surveys (gray symbols): LAEs at z ~ 4.9 (inverse triangles: Ouchi et al. 2003), LBGs at z ~ 5 (crosses and solid line: 
Yoshida et al. 2006), LAEs at z = 3.1 (open diamonds and dotted line: Ouchi et al. 2008), z = 3.7 (open boxes and dashed line: Ouchi et 
al. 2008), and z = 5.7 (open circle and dot-dash line: Ouchi et al. 2008). 




Fig. 12. — Number density of LAEs at Myv = —21.5 as a function of z. Our data point is shown with filled circles with a error bar. 
Open circles show the number densities derived by Ouchi et al. (2008). 




Fig. 13. — Left: Angular two-point correlation function (ACF) of our LAE sample. Filled circles show the ACF for the whole sample. 
The ACF for different quadrants arc shown with boxes, diamonds, circles, and crossed for the NE, NW, SW and SE subfield, respectively. 
Right: Same as the left panel with w(8) shown in logarithmic scale. 



